Operationalizing mobile media non-use with logging data in experience sampling studies

Julius Klingelhoefer

FAU Erlangen-Nürnberg

Alicia Gilbert

JGU Mainz

Christoph Adrian

FAU Erlangen-Nürnberg

Adrian Meier

FAU Erlangen-Nürnberg

06.12.2023

Background

Motivation

  • Self-reports - esp. about media use - can be biased (Parry et al., 2021)

  • Connectivity norms may influence self-reports (Geber et al., 2023)

  • Trace data could provide superior estimates (reliability, inobstrusive, resolution)

    • Logged app use

    • Notifications

    • …

“Objective” (Non-)use Parameters

Problem

  • Usually we don’t want to predict raw log data
  • Rather, measure psychological concepts, e.g.:
    • Distraction
    • Digital disconnection
    • Habits
    • …
  • Behavioral and cognitive-emotional components
  • Accessible for device and application level
  • Obstacles in assessing interaction & message level (Pouwels et al., 2023)

(Meier & Reinecke, 2021)

Approaches

  • Using category measures

    • e.g., screentime, number of apps

  • Generating/aggregating alternative measures

  • …

Our Study

Description

  • Two-week experience sampling study
  • N = 178 Android users
  • T = 7,823 disconnection questionnaires (disconnection, media use, well-being)
  • 8,268,433 log events (app started, screen on/off, notification)

Issues

Digital Disconnection Self-Report

Question: Whether disconnection happened at these levels in the last 2h.

Logged measures: Screentime

y r 95%-CI Lower 95%-CI Upper p
General disconnection - .16 - .18 - .14 < .001
  Device - .16 - .18 - .14 < .001
  Application - .15 - .17 - .13 < .001
  Feature - .12 - .14 - .09 < .001
  Interaction - .10 - .12 - .07 < .001
  Content - .09 - .11 - .07 < .001

Correlation with logged screentime, N = 7598

Disconnection as Substitution

com . android . chrome device application feature interaction content 0 50 100 150 200
device application feature interaction content 0 50 100 150 200 com . netflix . mediaclient

Discussion

  • ➡️ Showed link between self-reported and logged disconnection

  • ➡️ Disconnection behavioral vs. cognitive components

  • ➡️ Disconnection as reduction vs. disconnection as substitution

  • Future directions

    • Can complex behaviors be assessed through logging data?

    • Explore potential aggregation avenues

    • Evaluate logging methods

  • ❓Should we assess such constructs with digital trace data, and if so, how?

Thank you!

Questions?

References

Geber, S., Nguyen, M. H., & Büchi, M. (2023). Conflicting NormsHow Norms of Disconnection and Availability Correlate With Digital Media Use Across Generations. Social Science Computer Review, 08944393231215457. https://doi.org/10.1177/08944393231215457
Meier, A., & Reinecke, L. (2021). Computer-mediated communication, social media, and mental health: A conceptual and empirical meta-review. Communication Research, 48(8), 1182–1209. https://doi.org/10.1177/0093650220958224
Parry, D. A., Davidson, B. I., Sewall, C. J. R., Fisher, J. T., Mieczkowski, H., & Quintana, D. S. (2021). A systematic review and meta-analysis of discrepancies between logged and self-reported digital media use. Nature Human Behaviour, 5(11), 1535–1547. https://doi.org/10.1038/s41562-021-01117-5
Pouwels, J. L., Araujo, T., Atteveldt, W. van, Bachl, M., & Valkenburg, P. M. (2023). Integrating communication science and computational methods to study content-based social media effects. Communication Methods and Measures, 0(0), 1–9. https://doi.org/10.1080/19312458.2023.2285766
Siebers, T., Beyens, I., & Valkenburg, P. M. (2023). The effects of fragmented and sticky smartphone use on distraction and task delay. Mobile Media & Communication, 20501579231193941. https://doi.org/10.1177/20501579231193941

Supplement

Logged measures: Number of Apps

y r 95%-CI Lower 95%-CI Upper p
General disconnection - .05 - .07 - .03 < .001
   Device - .04 - .06 - .01 .003
   Application - .05 - .07 - .02 < .001
   Feature - .04 - .07 - .02 .001
   Interaction - .04 - .06 - .01 .002
   Content - .02 - .05  .00 .062

Correlation with logged screentime, N = 7598

WI Digital Methods Colloquium 2023
Julius Klingelhoefer, Alicia Gilbert, Christoph Adrian, & Adrian Meier